Information processing apparatus and method

ABSTRACT

The present disclosure relates to an information processing apparatus and method that allow an image projection position to be controlled with more ease. Correction information of an image is generated such that the image is projected onto a desired region of a real space. For example, the correction information is generated such that the projected image is located at a desired position as seen from a given viewpoint position. Also, for example, a correction vector is generated that corrects each pixel position of the image as the correction information. The present disclosure is applicable, for example, to an information processing apparatus, a projection apparatus, an imaging apparatus, a projection imaging apparatus, a projection imaging control apparatus, an image projection imaging system, or the like.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2019/014921 filed on Apr. 4, 2019, which claimspriority benefit of Japanese Patent Application No. JP 2018-079129 filedin the Japan Patent Office on Apr. 17, 2018. Each of theabove-referenced applications is hereby incorporated herein by referencein its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatusand method and particularly to an information processing apparatus andmethod that allow an image projection position to be controlled withmore ease.

BACKGROUND ART

To date, there have been conceived a wide variety of techniques forcontrolling projection of an image onto a desired position in imageprojection using a projector. In particular, what is called anultra-short focus projector projects an image from a location closer toa projection plane than a conventional long focus projector, more likelyresulting in larger distortion of the projected image than the longfocus projector and involving a high level of difficulty in imageprojection onto a desired position (within desired bounds).

As such an image correction technique, there has been one conceived forcorrecting an image so as to be projected onto a desired position byemitting a special pattern, causing a user to capture an image of thepattern with a camera in hand, and estimating distortion of a projectedimage on the basis of the captured image (refer, for example, to PTL 1).

CITATION LIST Patent Literature

[PTL 1]

Japanese Patent Laid-open No. 2013-192098

SUMMARY Technical Problem

However, such a technique requires troublesome tasks including captureof an image with a camera in hand by a user or the like.

The present disclosure has been devised in light of the foregoingcircumstances, and it is an object of the present disclosure to controlan image projection position with more ease.

Solution to Problem

An information processing apparatus of an aspect of the presenttechnology includes a correction information generation section adaptedto generate correction information of an image such that the image isprojected onto a desired region of a real space.

An information processing method of an aspect of the present technologyincludes generating correction information of an image such that theimage is projected onto a desired region of a real space.

In the information processing apparatus and method of the aspect of thepresent technology, correction information of an image is generated suchthat the image is projected onto a desired region of a real space.

Advantageous Effect of Invention

According to the present disclosure, it is possible to processinformation. In particular, it is possible to control an imageprojection position with more ease.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example of main components ofa projection imaging system.

FIG. 2 is a block diagram illustrating an example of main components ofa projection imaging apparatus.

FIG. 3 is a functional block diagram illustrating an example of mainfunctions realized by a control section.

FIG. 4 is a flowchart describing an example of a flow of a projectioncorrection process.

FIG. 5 is a flowchart describing an example of a flow of a screen frameposition detection process.

FIG. 6 is a diagram illustrating an example of an image for detecting ascreen frame position and an example of a captured image of a screenframe position.

FIG. 7 is a diagram illustrating an example of a manner in which ascreen frame position is detected.

FIG. 8 is a diagram illustrating an example of a manner in which ascreen frame position is detected.

FIG. 9 is a diagram illustrating an example of a manner in which ascreen frame position is detected.

FIG. 10 is a flowchart describing an example of a flow of a correctionvector generation process.

FIG. 11 is a diagram illustrating an example of a manner in which acorrection vector is generated.

FIG. 12 is a diagram illustrating an example of a manner in which acorrection vector is generated.

FIG. 13 is a diagram illustrating an example of a manner in which acorrection vector is generated.

FIG. 14 is a diagram illustrating an example of a manner in which acorrection vector is generated.

FIGS. 15A and 15B illustrate diagrams of an example of a manner in whichan image is corrected.

FIGS. 16A and 16B illustrate block diagrams of examples of maincomponents of the projection imaging apparatus.

FIG. 17 is a block diagram illustrating an example of main components ofa projection imaging system.

DESCRIPTION OF EMBODIMENTS

Modes for carrying out the present disclosure (hereinafter referred toas embodiments) will be described below. It should be noted that thedescription will be given in the following order.

1. Image projection position control

2. First embodiment (projection imaging system)

3. Second embodiment (another configuration example)

4. Note

1. Image Projection Position Control

<Image Projection>

To date, there have been conceived a wide variety of techniques forcontrolling projection of an image onto a desired position in imageprojection using a projector. In particular, what is called anultra-short focus projector, which has recently been used widely,projects an image from a location closer to a projection plane than aconventional long focus projector, more likely resulting in largerdistortion of the projected image than the long focus projector andinvolving a high level of difficulty in image projection onto a desiredposition (within desired bounds).

As such an image correction technique, there has been one conceived forcorrecting an image so as to be projected onto a desired position byemitting a special pattern, causing a user to capture an image of thepattern with a camera in hand, and estimating distortion of a projectedimage on the basis of the captured image (refer, for example, to PTL 1).

In the case of such a technique, however, troublesome tasks includingcapture of an image with a camera in hand by a user or the like arerequired. Also, the technique described in PTL 1 only correctsdistortion of a projected image. In order to project an image onto adesired position (place a projected image at a desired position),troublesome tasks including manually adjusting a projector's projectiondirection and the like are also required. As a result, an easy-to-usetechnique for controlling the image projection position has beendemanded.

<Image Projection Position Control>

For this reason, image correction information is generated to ensurethat the image is projected within a desired region of a real space.

For example, a correction information generation section for generatingcorrection information of an image is provided in an informationprocessing apparatus to project the image within a desired region of thereal space.

This makes it possible to control the image projection position withmore ease by simply correcting an image to be projected with thecorrection information.

2. First Embodiment

<Projection Imaging System>

FIG. 1 is a block diagram illustrating an example of main components ofone embodiment of a projection imaging system to which the presenttechnology is applied. In FIG. 1, a projection imaging system 100includes a projection imaging apparatus 101 and a screen 102. Theprojection imaging system 100 is a system that projects an image,captures an image of a projection plane, and controls an imageprojection position (bounds).

The projection imaging apparatus 101 is an apparatus that projects animage onto the screen 102. The projection imaging apparatus 101 isinstalled close to the screen 102 as is what is called an ultra-shortfocus projector and is designed to project an image from there onto thescreen 102. For example, the projection imaging apparatus 101 isinstalled at a position under the screen 102 in contact or in proximityto a wall or the like to which the screen 102 is installed so as toproject an image toward the screen 102 (wall) from a location at aslight distance from a housing thereof. That is, the projection imagingapparatus 101 projects an image from immediately under the screen 102.

Also, the projection imaging apparatus 101 controls the position where(the bounds within which) an image is projected. The projection imagingapparatus 101 corrects the image to be projected so as to control theimage projection position. Further, the projection imaging apparatus 101captures an image of the screen 102 onto which the image is projected(i.e., projection plane) to correct the image (control the projectionposition). In general, an image projected by a short focus projector isprone to distortion, and it is highly difficult to control the positionthereof. The projection imaging apparatus 101 achieves image projectionposition control by capturing an image of the screen 102 and performingimage correction by using the captured image, thus allowing the imageprojection position to be controlled with more ease.

It should be noted that the screen 102 includes a material orconfiguration that allows reflection of light forward with highluminance and in other directions with low luminance. For example, inthe case where an image projected by the projection imaging apparatus101 onto the screen 102 is viewed from front, the projected image lookshighly bright. However, in the case where the projected image is viewedfrom directions other than the front (e.g., from under the screen 102),the projected image looks dim. The screen 102 realizes the projectionplane having such a characteristic with its material or configuration.

<Projection Imaging Apparatus>

FIG. 2 is a block diagram illustrating an example of main components ofthe projection imaging apparatus 101 that is one embodiment of theinformation processing apparatus to which the present technology isapplied. It should be noted that only main processing sections, dataflows, and the like are depicted in FIG. 2 and that not all thecomponents of the projection imaging apparatus 101 are necessarilydepicted in FIG. 2. That is, in the projection imaging apparatus 101,there may be processing sections not depicted as blocks in FIG. 2, andthere may be processes and data flows not depicted as arrows and thelike in FIG. 2.

As illustrated in FIG. 2, the projection imaging apparatus 101 includesa projection imaging unit 111 and a control unit 112. The projectionimaging unit 111 handles processes associated with projection and imagecapture. The control unit 112 handles processes associated with controlover the projection imaging unit 111.

The projection imaging unit 111 includes a projection section 121 andimaging sections 122-1 and 122-2.

The projection section 121 handles processes associated with imageprojection. For example, the projection section 121 acquires image datafrom the control unit 112 (control section 131 described later) andprojects the image thereof onto the screen 102.

The imaging sections 122-1 and 122-2 handle processes associated withimage capture. In the case where there is no need to distinguish betweenthe imaging sections 122-1 and 122-2, the two will be referred to as theimaging sections 122. For example, the imaging sections 122 generatecaptured image data by capturing images of the screen 102 onto which animage is projected by the projection section 121 and its periphery. Theimaging sections 122 supply the generated captured image data to thecontrol unit 112 (control section 131 described later). It should benoted that the imaging sections 122-1 and 122-2 capture images of thescreen 102 and its periphery from different positions in the housing ofthe projection imaging apparatus 101. That is, there is parallax betweena captured image generated by the imaging section 122-1 and a capturedimage generated by the imaging section 122-2.

The control unit 112 includes the control section 131, an input section141, an output section 142, a storage section 143, a communicationsection 144, and a drive 145.

The control section 131 handles processes associated with control overthe projection imaging unit 111. For example, the control section 131supplies image data to the projection section 121, thus causing theimage thereof to be projected. Also, the control section 131 acquirescaptured image data by causing the imaging section 122 to capture animage.

Further, the control section 131 corrects the image to be projected bythe projection section 121 by use of the captured image, thuscontrolling the position (bounds) of the image projected by theprojection section 121. For example, the control section 131 controlsthe image projection position (bounds) such that the bounds of the imageprojected by the projection section 121 match those of the screen 102(such that an outer frame of the projected image matches that of thescreen 102).

It should be noted that although configured in any manner, the controlsection 131 may include a CPU (Central Processing Unit), a ROM (ReadOnly Memory), a RAM (Random Access Memory), and the like and carry outprocesses as the CPU loads programs and data stored in the ROM or thelike into the RAM for execution, for example.

The input section 141 handles processes associated with receipt ofinformation input from external equipment. For example, the inputsection 141 receives information input from external equipment andsupplies the information to the control section 131. The input section141 includes any input device that receives external information such asuser input. The input device may be, for example, a keyboard, a mouse,an operating button, a touch panel, a camera, a microphone, or an inputterminal. Also, the input device may be one of a variety of sensors suchas an acceleration sensor, an optical sensor, and a temperature sensor,or input equipment such as a barcode reader. The input section 141 mayinclude a plurality of input devices or a plurality of types of inputdevices.

The output section 142 handles processes associated with informationoutput. For example, the output section 142 outputs information suppliedfrom the control section 131 to equipment external to the projectionimaging apparatus 101. The output section 142 includes any output devicethat outputs information such as images and sounds. The output devicemay be, for example, a display, a speaker, an output terminal, or thelike. The output section 142 may include a plurality of output devicesor a plurality of types of input devices.

The storage section 143 handles processes associated with informationstorage. For example, the storage section 143 stores informationsupplied from the control section 131. Also, the storage section 143supplies information stored therein to the control section 131. Thestorage section 143 includes any storage medium that stores informationsuch as programs and data. The storage medium may be, for example, ahard disk, a RAM disk, a non-volatile memory, or the like. The storagesection 143 may include a plurality of storage media or a plurality oftypes of storage media.

The communication section 144 handles processes associated withcommunication. For example, the communication section 144 communicateswith an apparatus external to the projection imaging apparatus 101,supplying information (e.g., programs and data) supplied from thecontrol section 131 to the external apparatus. Also, the communicationsection 144 communicates with an apparatus external to the projectionimaging apparatus 101, acquiring information (e.g., programs and data)from the external apparatus and supplying the acquired information tothe control section 131. The communication section 144 includes anycommunication device for communicating with an external apparatus via agiven communication medium (e.g., any network such as the Internet) andexchanging information such as programs and data. The communicationdevice may be, for example, a network interface. The communicationsection 144 may include a plurality of communication devices or aplurality of types of communication devices. It should be noted that thecommunication section 144 may have a wired or wireless communicationfunction or both thereof.

The drive 145 handles processes associated with an interface of aremovable medium 151 inserted therein. For example, the drive 145 readsout information (e.g., programs and data) that is recorded in theremovable medium 151 inserted therein, supplying the information to thecontrol section 131. Also, in the case where the writable removablemedium 151 is inserted in the drive 145, the drive 145 writes (stores)information (e.g., programs and data) supplied from the control section131 to (in) the removable medium 151. The optional removable medium 151may be, for example, a magnetic disk, an optical disc, a magneto-opticaldisk, a semiconductor memory, or the like. The drive 145 may allowinsertion of the plurality of removable media 151 or the plurality oftypes of removable media 151.

<Functional Blocks of the Control Section>

FIG. 3 is a functional block diagram illustrating an example offunctions realized as a result of execution of programs by the controlsection 131. As illustrated in FIG. 3, the control section 131 provides,for example, functions of a projection control section 201, an imagingcontrol section 202, a corresponding point detection section 211, ascreen frame position detection section 212, a posture estimationsection 213, a screen shape estimation section 214, and a correctionvector generation section 215 by executing programs.

The projection control section 201 handles processes associated withcontrol over the projection section 121. For example, the projectioncontrol section 201 supplies image data to the projection section 121,thus causing the image thereof to be projected. For example, in the casewhere the user views content, the projection control section 201supplies image data of the content to the projection section 121, thuscausing the image thereof to be projected. Also, in the case ofdetecting corresponding points between the projection section 121 andthe imaging section 122 (corresponding points between an image to beprojected and a captured image), the projection control section 201supplies image data for detecting the corresponding points to theprojection section 121 in response to a request from the correspondingpoint detection section 211, thus causing the image thereof to beprojected. Also, in the case of detecting a frame position of the screen102, the projection control section 201 supplies image data fordetecting the screen frame position to the projection section 121 inresponse to a request from the screen frame position detection section212 (frame detection imaging processing section 221), thus causing theimage thereof to be projected.

Also, for example, the projection control section 201 subjects the imageto be supplied to the projection section 121 to a correction process forcontrolling the image projection position or correcting distortion ofthe projected image. For example, the projection control section 201performs the correction process by using a correction vector suppliedfrom the correction vector generation section 215 (correction vectorcalculation section 233).

The imaging control section 202 handles processes associated withcontrol over the imaging section 122. For example, in the case ofdetecting corresponding points between the projection section 121 andthe imaging section 122 (corresponding points between an image to beprojected and a captured image), the imaging control section 202controls, in response to a request from the corresponding pointdetection section 211, the imaging section 122 to capture an image ofthe screen 102 onto which an image is projected from the projectionsection 121 and its periphery and generate captured image data, thusacquiring the data. The imaging control section 202 supplies theacquired captured image data to the corresponding point detectionsection 211. Also, in the case of detecting the frame position of thescreen 102, the imaging control section 202 controls, in response to arequest from the screen frame position detection section 212 (framedetection imaging processing section 221), the imaging section 122 tocapture an image of the screen 102 onto which an image is projected fromthe projection section 121 and its periphery and generate captured imagedata, thus acquiring the data. The imaging control section 202 suppliesthe acquired captured image data to the screen frame position detectionsection 212 (frame detection imaging processing section 221).

The frame detection imaging processing section 221 handles processesassociated with detection of corresponding points between the projectionsection 121 and the imaging section 122 (corresponding points between animage to be projected and a captured image). For example, the framedetection imaging processing section 221 controls the projection section121 via the projection control section 201, supplying image data forcorresponding point detection and causing the image thereof to beprojected. Also, the frame detection imaging processing section 221controls the imaging section 122 via the imaging control section 202,causing the imaging section 122 to capture an image of the screen 102onto which the image for corresponding point detection is projected andits periphery and acquiring captured image data thereof.

The corresponding point detection section 211 detects correspondingpoints by using the acquired captured image data and the like, supplyingthe detection result thereof to the posture estimation section 213 and aprojected image frame position identification section 224 of the screenframe position detection section 212.

The screen frame position detection section 212 handles processesassociated with detection of a screen frame position. For example, thescreen frame position detection section 212 detects the frame positionof the screen 102, supplying the detection result thereof to thecorrection vector generation section 215 (viewpoint image frame positioncalculation section 232).

The posture estimation section 213 handles processes associated withposture estimation. For example, the posture estimation section 213estimates the postures of the projection section 121 and the imagingsection 122 by using the corresponding points detected by thecorresponding point detection section 211, supplying the estimationresult thereof to the screen shape estimation section 214.

The screen shape estimation section 214 handles processes associatedwith screen shape estimation. For example, the screen shape estimationsection 214 estimates the shape of the screen 102 on the basis of thepostures of the projection section 121 and the imaging section 122estimated by the posture estimation section 213, supplying theestimation result thereof to the correction vector generation section215 (viewpoint position estimation section 231).

The correction vector generation section 215 handles processesassociated with generation of a correction vector. The correction vectoris vector information indicating details of image correction forcontrolling the image projection position (bounds) and reducingdistortion. That is, a correction vector is information indicating howto correct each pixel of the image to be projected by the projectionsection 121 so as to control the image projection position (bounds) andreduce distortion. For example, the correction vector generation section215 generates a correction vector on the basis of information such asthe screen shape estimated by the screen shape estimation section 214,the frame position of the screen 102 detected by the screen frameposition detection section 212, and the like.

The correction vector generation section 215 supplies the generatedcorrection vector to the projection control section 201. As describedabove, the projection control section 201 performs image correction byusing the correction vector.

Also, the screen frame position detection section 212 includes the framedetection imaging processing section 221, a lens distortion correctionsection 222, a captured image frame position detection section 223, theprojected image frame position identification section 224, and a frameposition interpolation process section 225.

The frame detection imaging processing section 221 handles processesassociated with image capture for detecting a screen frame. For example,the frame detection imaging processing section 221 controls theprojection section 121 via the projection control section 201, supplyingimage data for detecting the screen frame position and causing the imagethereof to be projected. Also, the frame detection imaging processingsection 221 controls the imaging section 122 via the imaging controlsection 202, causing the imaging section 122 to capture an image of thescreen 102 onto which the image for detecting the screen frame positionis projected from the projection section 121 and its periphery andgenerate captured image data and acquiring the data thereof. Also, theframe detection imaging processing section 221 supplies the acquiredcaptured image data to the lens distortion correction section 222.

The lens distortion correction section 222 handles processes associatedwith lens distortion correction. For example, the lens distortioncorrection section 222 corrects lens distortion of the captured imagesupplied from the frame detection imaging processing section 221,supplying the corrected captured image to the captured image frameposition detection section 223.

The captured image frame position detection section 223 handlesprocesses associated with screen frame position detection in a capturedimage. For example, the captured image frame position detection section223 detects, by using a captured image whose lens distortion has beencorrected by the lens distortion correction section 222, the frameposition of the screen 102 in the captured image, supplying thedetection result thereof to the projected image frame positionidentification section 224.

The projected image frame position identification section 224 handlesprocesses associated with screen frame position detection in an image tobe projected. For example, the projected image frame positionidentification section 224 transforms the frame position of the screen102 in the captured image that is detected by the captured image frameposition detection section 223 into a frame position in the image to beprojected, by use of the corresponding points detected by thecorresponding point detection section 211, thus identifying the frameposition of the screen 102 in the image to be projected. The projectedimage frame position identification section 224 supplies informationindicating the identified frame position of the screen 102 to the frameposition interpolation process section 225.

The frame position interpolation process section 225 handles processesassociated with interpolation of the screen frame position identified bythe projected image frame position identification section 224. Forexample, the frame position interpolation process section 225 performsinterpolation between local screen frame positions identified by theprojected image frame position identification section 224 by a giveninterpolation technique. The frame position interpolation processsection 225 supplies, to the correction vector generation section 215(viewpoint image frame position calculation section 232), informationindicating the frame position of the screen 102 after the interpolationprocess.

The correction vector generation section 215 includes the viewpointposition estimation section 231, the viewpoint image frame positioncalculation section 232, and the correction vector calculation section233.

The viewpoint position estimation section 231 handles processesassociated with viewpoint position estimation. For example, theviewpoint position estimation section 231 estimates a user's viewpointposition (e.g., front side of the screen 102) on the basis of the screenshape estimated by the screen shape estimation section 214. Theviewpoint position estimation section 231 supplies informationindicating the estimated position to the viewpoint image frame positioncalculation section 232.

The viewpoint image frame position calculation section 232 handlesprocesses associated with calculation of a screen frame position in aviewpoint image indicating how much is visible from the viewpointposition (field of view). For example, the viewpoint image frameposition calculation section 232 estimates a viewpoint image at theviewpoint position estimated by the viewpoint position estimationsection 231 and calculates, with respect to the viewpoint image, theframe position of the screen 102 indicated in information supplied fromthe frame position interpolation process section 225, supplying thecalculation result thereof to the correction vector calculation section233.

The correction vector calculation section 233 handles processesassociated with correction vector calculation. For example, thecorrection vector calculation section 233 calculates a correction vectorby using the frame position of the screen 102 in the viewpoint imagecalculated by the viewpoint image frame position calculation section232. The correction vector calculation section 233 supplies thecalculated correction vector to the projection control section 201.

<Flow of the Projection Correction Process>

A description will be given next of an example of a flow of a projectioncorrection process performed by the projection imaging apparatus 101configured as described above with reference to the flowchart in FIG. 4.It should be noted that the process may be performed during projectionof content or when no content is projected.

When the projection correction process begins, the corresponding pointdetection section 211 detects corresponding points between a projectedimage and a captured image in step S101. Any technique may be used todetect the corresponding points. For example, the technique disclosed inPCT Patent Publication No. WO2017/104447 may be used. It should be notedthat content viewing may be suspended when corresponding points aredetected so as to acquire the corresponding points by emitting astructured light pattern.

In step S102, the screen frame position detection section 212 detectsthe frame position of the screen 102.

In step S103, the posture estimation section 213 estimates the posturesof the projection section 121 and the imaging section 122 by using thecorresponding points acquired in step S101.

In step S104, the screen shape estimation section 214 estimates thescreen shape on the basis of the posture estimation result in step S103.

In step S105, the correction vector generation section 215 estimates acorrection vector by using the screen shape estimated in step S104.

When the process in step S105 ends, the projection correction processends.

<Flow of the Screen Frame Position Detection Process>

A description will be given next of an example of a flow of the screenframe position detection process performed in step S102 in FIG. 4 withreference to the flowchart in FIG. 5.

When the screen frame position detection process begins, the framedetection imaging processing section 221 selects the target imagingsection 122 in step S121. In the case where there is only one imagingsection 122, the process is omitted.

In step S122, the frame detection imaging processing section 221 causesthe projection section 121 to project an image for detecting a screenframe position and the imaging section 122 to capture an image of theprojection plane thereof, thus generating the captured image fordetecting a screen frame position.

FIG. 6 illustrates an example of a projected image for detecting ascreen frame position and an example of a captured image for detecting ascreen frame position. A projected image 301 for detecting a screenframe position illustrated in FIG. 6 depicts an example of an imageprojected by the projection section 121. In this case, the projectedimage 301 for detecting a screen frame position is what is called awhite image all of whose pixels are set to a maximum luminance level.That is, the projection section 121 projects such a uniformly whiteimage as a projected image for detecting a screen frame position.

It should be noted that the projected image 301 for detecting a screenframe position is not limited in luminance level to the example in FIG.6. For example, it is acceptable as long as the projected image 301 fordetecting a screen frame position has a uniform luminance level, andthere is no need for the projected image 301 to have the maximumluminance level. That is, the projected image 301 for detecting a screenframe position may be an image of a color other than white. Also, it isacceptable as long as the projected image 301 for detecting a screenframe position has a uniform luminance level at its exterior (in thevicinity of the frame), and there is no need for the projected image 301to have a uniform luminance level in the center.

The imaging section 122 captures an image of the screen 102 onto whichthe projected image 301 for detecting a screen frame position isprojected and its periphery, thus acquiring a captured image 302 fordetecting a screen frame position.

A projected image 311 included in the captured image 302 for detecting ascreen frame position is the projected image 301 for detecting a screenframe position projected onto the screen 102 or the like by theprojection section 121. A region 312 in the projected image 311 is lowerin luminance than its periphery. The reason for this is that the screen102 reflects light forward with high luminance and that less light isreflected in other directions (in the direction where the projectionimaging apparatus 101 is installed) than by the periphery of the screen102.

The projection imaging apparatus 101 detects the frame of the screen 102by using such a difference in luminance. That is, the projection imagingapparatus 101 identifies the frame position of the screen 102 by usingthe difference between a light intensity with which an all-white patternemitted from the projection section 121 enters the imaging section 122after having been reflected by the screen 102 and a light intensity withwhich the pattern enters the imaging section 122 after having beenreflected by the exterior of the screen 102 (e.g., wall).

It should be noted that the imaging section 122 captures images with awide-angle lens to ensure that the entire screen 102 fits within animaging angle of view. For this reason, the captured image 302 fordetecting a screen frame position has lens distortion, changing astraight line in a real space into a curve in the captured image 302 fordetecting a screen frame position.

For this reason, the lens distortion correction section 222 applies, instep S123, lens distortion correction to the captured image 302 fordetecting a screen frame position generated in step S122, by using aknown lens distortion parameter from advance calibration, thus reducingsuch lens distortion. Further, the lens distortion correction section222 transforms the image into a gray image.

A captured image 321 illustrated in FIG. 7 is acquired as a result ofthe processes. A projected image 322 in the captured image 321corresponds to the projected image 311 (FIG. 6) of the captured image302 for detecting a screen frame position. Also, a region 323 with lowerluminance than its periphery in the projected image 322 corresponds tothe region 312 in the captured image 302 for detecting a screen frameposition. As described above, curved contours of the projected image 311and the region 312 are transformed into straight lines as a result oflens distortion correction.

In step S124, the captured image frame position detection section 223detects a screen frame position in the captured image.

More specifically, the captured image frame position detection section223 binarizes the captured image 321 first, thus transforming the imageinto a binary image 331. In the binary image 331, only a region outsidethe screen 102 onto which the projected image 301 for detecting a screenframe position is projected, the region having high luminance in thecaptured image 321, is represented as a black region 332 as illustratedin FIG. 7. That is, such binarization clarifies the difference inluminance between the region of the screen 102 and the region outsidethe screen 102.

Next, the captured image frame position detection section 223 performscontour detection in the binary image 331, thus detecting a contour ofthe black region 332. An image 341 in FIG. 7 depicts an example of aresult of the contour detection. As illustrated in the image 341, anouter contour (dotted line frame 342) and an inner contour (solid lineframe 343) of the black region 332 are both detected. The dotted lineframe 342 corresponds to the contour of the projected image 301 fordetecting a screen frame position. Also, the solid line frame 343corresponds to the contour of the screen 102.

Next, the captured image frame position detection section 223 identifiesthe contour of the screen 102 in the image 341 on the basis of a lengthof the contour of the screen 102 that is approximately known in advance.That is, the solid line frame 343 is identified. An image 351 in FIG. 7depicts the identification result. A contour 352 in the image 351corresponds to the solid line frame 343 of the image 341.

The frame position of the screen 102 in a captured image is detected asdescribed above.

In step S125, the projected image frame position identification section224 identifies the screen frame position in the projected image by usingthe corresponding points detected in step S101 in FIG. 4. That is, theprojected image frame position identification section 224 transforms theframe position of the screen 102 in the captured image detected asdescribed above into a frame position in the projected image, by usingthe corresponding points.

First, the projected image frame position identification section 224regards the detected contour 352 as a set of points (also referred to ascontour points), identifying a corresponding point 362 on the capturedimage closest to each of the contour points and linking the contourpoints to the corresponding point 362 as illustrated in an image 361 inFIG. 8.

An image 371 in FIG. 8 is an enlarged image of a partial region 363 ofthe image 361. It is assumed, as illustrated in the image 371, thatthere are corresponding points 362-1, 362-2, and 362-3 in the vicinityof the contour 352. The corresponding point 362-1 is closest to thecontour points within bounds 372-1, with respect to the partial contour352 in the image 371. Therefore, the contour points are linked to thecorresponding point 362-1. Also, the corresponding point 362-2 isclosest to the contour points within bounds 372-2. Therefore, thecontour points are linked to the corresponding point 362-2. Similarly,the corresponding point 362-3 is closest to the contour points withinbounds 372-3. Therefore, the contour points are linked to thecorresponding point 362-3.

Next, the projected image frame position identification section 224detects corresponding points in the periphery of the linkedcorresponding point, grouping the points together. As illustrated in animage 381 in FIG. 8, for example, the corresponding point 362-1, thecorresponding point 362-3, a corresponding point 362-4, a correspondingpoint 362-5, and a corresponding point 362-6 are grouped together inrelation to the corresponding point 362-2 linked to a contour pointgroup 352-2.

The projected image frame position identification section 224 obtainshomographic transformation between the captured image and the projectedimage by using the grouped corresponding points. Then, the projectedimage frame position identification section 224 applies the homographictransformation to each of the contour points linked to the correspondingpoint, thus transforming the contour points into coordinates of aprojected image.

For example, in the case of the image 381, the projected image frameposition identification section 224 obtains its homographictransformation by using the grouped corresponding points 362-1 to 362-6.Then, the projected image frame position identification section 224applies the homographic transformation to the contour point group 352-2linked to the corresponding point 362-2, thus transforming the contourpoints into coordinates of a projected image.

The projected image frame position identification section 224 transformsthe contour 352 into a contour in a projected image by applying such aprocess to all the contour points. An image 391 in FIG. 8 depicts anexample of a frame position of the screen 102 of the projected imageobtained in this manner. That is, the image 391 corresponds to theprojected image, and a solid line frame 392 corresponds to the contour352 transformed into the contour in the projected image.

In step S126, the frame position interpolation process section 225performs model fitting on the screen frame position in the projectedimage identified in step S125, thus conducting interpolation betweenscreen frame positions.

As described above, the transformation of the contour 352 into a contourin a projected image is carried out for each local portion, possiblychanging the solid line frame 392 into a jaggy contour 401 (contour withdiscontinuities) as illustrated in FIG. 9 due to noise impact or thelike. For this reason, the frame position interpolation process section225 performs model fitting on the screen frame position, thus conductinginterpolation between screen frame positions and transforming thecontour 401 into a contour 402. Any technique may be used forinterpolation. For example, B-spline interpolation may be applied. It ispossible to keep, to a minimum, noise impact by applying such aninterpolation technique and connecting the screen frame positionssmoothly.

In step S127, the screen frame position detection section 212 determineswhether or not the process is complete for all the imaging sections 122.In the case where it is determined that there are still the imagingsections 122 yet to be processed, the process returns to step S121 torepeat the subsequent processes. That is, the above processes arerepeated for each imaging section 122.

Then, in the case where it is determined in step S127 that the processis complete for all the imaging sections 122, the process proceeds tostep S128.

In step S128, the screen frame position detection section 212 combinesthe frame positions of the screen 102 in the projected image detected onthe basis of the respective captured images. Any technique may be usedto combine the frame positions. For example, a mean of the respectiveframe positions of the screen 102 detected in the projected image or amedian thereof may be used as a final frame position.

When the process in step S128 ends, the screen frame position detectionprocess ends, and the process returns to FIG. 4.

<Flow of the Correction Vector Generation Process>

A description will be given next of an example of a flow of a correctionvector generation process performed in step S105 in FIG. 4 withreference to the flowchart in FIG. 10.

When the correction vector generation process begins, the viewpointposition estimation section 231 of the correction vector generationsection 215 estimates a viewpoint image at a viewpoint position in stepS141.

The viewpoint image refers to an image indicating how much is visiblefrom a viewpoint position (field of view), i.e., a virtual imageequivalent to a captured image obtained by capturing an image of thescreen 102 and its periphery from the viewpoint position. Also, theviewpoint position refers to a viewpoint position of a user viewing theprojected image, and any position can be specifically set as a viewpointposition. It is assumed here that the viewpoint position is locatedforward from the screen 102 at a given distance.

A correction vector is calculated by using such a viewpoint image. Thatis, the projection imaging apparatus 101 calculates the correctionvector such that the position (bounds) of the projected image as seenfrom the user agrees with the position (bounds) of the screen 102.

In step S142, the viewpoint image frame position calculation section 232calculates the screen frame position in the viewpoint image. Asdescribed above, the frame position of the screen 102 in each of thecaptured images and each of the projected images is obtained. This makesit possible to obtain the frame position of the screen 102 in theviewpoint image.

In step S143, the correction vector calculation section 233 calculates acorrection vector in consideration of plane model misalignment andscreen frame distortion. Because of a potential risk of errorattributable to actual environmental impacts, the correction vectorcalculation section 233 calculates the correction vector inconsideration of such an error.

For example, as illustrated in FIG. 11, the estimated position of thescreen 102 (plane model) may be misaligned from the actual position.Such an error (plane model misalignment) may lead to a differencebetween the position of a certain pixel 421 in the projected imageassumed on an estimated plane and the position of an actual measuredpoint 422 that has actually been measured.

For this reason, the correction vector calculation section 233calculates a correction vector to reduce such a plane modelmisalignment.

Also, the frame position of the screen 102 in the projected imageobtained as described above may become distorted (screen framedistortion) as does a frame position 432 of a projected image 431illustrated in FIG. 12.

For this reason, the correction vector calculation section 233calculates a correction vector such that distortion diminishes asdepicted by a frame position 442 in a viewpoint image 441.

At this time, the correction vector calculation section 233 calculates acorrection vector not only to tidy up the shape of the frame (outershape of the projected image) but also to correct the distortion of theprojected image as a whole.

For example, it is assumed that the frame position of a projected imageis distorted as illustrated within the dotted line frame in FIG. 13. Inthe case of correcting the distortion of the image as illustrated inFIG. 14, the correction vector calculation section 233 corrects thedistortion, for example, by applying interpolation within a triangularelement. For example, the correction vector calculation section 233 setspoints p1 and p2 relative to an origin p0 as illustrated in FIG. 13 andcarries out linear interpolation for a point p inside a regionsurrounded by the three points (rectangular element).

A local coordinate system that includes a triangle having its origin ata point p0 is obtained by a basis vector e_(u) and a basis vector e_(v)as given by the following formulas (1) and (2), and a transformationmatrix to a global coordinate system is given by the following formula(3) in which u_(p) and v_(p) represent the coordinates of the localcoordinate system of a point p and x_(p), y_(p), and z_(p) represent thecoordinates of the global coordinate system.[Math. 1]e _(u) =p ₁ −p ₀  (1)e _(v) =p ₂ −p ₀  (2)p=p ₀ +u _(p) e _(u) +v _(p) e _(v)  (3)

Also, the coordinates are expressed by their components as given by thefollowing formula (4).

[Math.  2] $\begin{matrix}{\begin{pmatrix}x_{p} \\y_{p} \\z_{p}\end{pmatrix} = {\begin{pmatrix}x_{p\; 0} \\y_{p\; 0} \\z_{p\; 0}\end{pmatrix} + {\begin{pmatrix}x_{eu} & x_{ev} \\y_{eu} & y_{ev} \\z_{eu} & z_{ev}\end{pmatrix}\begin{pmatrix}u_{p} \\v_{p}\end{pmatrix}}}} & (4)\end{matrix}$

Letting respective values at nodes p0, p1, and p2 be denoted as C₀, C₁,and C₂, a value C at the point p in the element can be expressed by theformula (5) given below.[Math. 3]C=(C ₀ +u _(p)(C ₁ −C ₀)+v _(p)(C ₂ −C ₀)  (5)

Here, u_(p) and v_(p) are the coordinates of the point p in the localcoordinate system. C represents an interpolation correction vectorvalue, and C₀-C₂ represents a correction vector value of p0-p2.

Needless to say, any technique may be used for interpolation, and theinterpolation technique is not limited to the above example.

In step S144, the correction vector calculation section 233 supplies thecorrection vector calculated as described above to the projectioncontrol section 201, thus reflecting the correction vector intoprojection control. That is, the projection control section 201 correctsthe image to be projected by the projection section 121, by using thecorrection vector.

When the process in step S144 ends, the correction vector generationprocess ends, and the process returns to FIG. 4.

In the case where the projected image position (bounds) is notcontrolled as described above, for example, similarly to a projectedimage 502 depicted in FIG. 15A, a projected image may lie off the screen501 or be misaligned, possibly degrading a subjective image quality(image quality as seen from the user) of the projected image 502.

In contrast, performing each of the processes described above allowsprojection to achieve alignment between the screen 501 and the position(bounds) of the projected image 502 with ease. This keeps, to a minimum,degradation of the subjective quality of the projected image 502. Also,a customer experience value can be improved from the viewpoint of designand the like. Further, the above correction allows for correction of notonly the frame position of the projected image but also internaldistortion of the image. This keeps, to a minimum, degradation of thesubjective quality of the projected image 502.

<Correction in the Middle of Content Viewing>

The correction as described above may be performed while content isprojected or when no content is projected. In the case where thecorrection is performed while content is projected, the correction ispreferably performed in such a manner as not to interfere with contentviewing.

In that case, for example, it is only necessary to project an image fordetecting a screen frame position by using the Imperceptible StructuredLight technique disclosed in PCT Patent Publication No. WO2017/104447.For example, it is only necessary to embed a uniform luminance patternin content and emit the pattern. In that case, the luminance value ofthe projected image for detecting a screen frame position may be reducedto a suitable level. This can render the projected image for detecting ascreen frame position, which is embedded in content, inconspicuous, thuskeeping, to a minimum, degradation of the subjective image quality ofcontent.

<Projected Image for Detecting a Screen Frame Position>

It should be noted that an image of any kind may be used as a projectedimage for detecting a screen frame position and that the luminance leveldoes not need to be uniform across the image. Also, any luminance levelmay be used, and a black image (image with a luminance level of 0) maybe used rather than a white image. Further, for example, a plurality ofimages with different luminance levels and patterns may be used.

<Correction Range>

Although a description has been given above regarding correction of aprojected image as a whole, the present disclosure is not limitedthereto, and only part of a projected image, for example, may becorrected. For example, the projection of an image from under the screenwith an ultra-short focus projector tends to cause distortion in theupper side of the screen. In such a case, a comparatively highlydistorted portion (part of the upper side of the screen) may besubjected to the above correction.

<Position of the Projection Imaging Apparatus>

Although a description has been given above regarding the projection ofan image by the projection imaging apparatus 101 from near the screen102 as with what is called an ultra-short focus projector, theprojection imaging apparatus 101 may be located at any position. Forexample, the projection imaging apparatus 101 may project an image at adistance from the screen 102 as with what is called a long-focusprojector.

3. Second Embodiment

<Another Configuration Example>

The projection imaging apparatus 101 to which the present technology isapplied is not limited in configuration to the example in FIG. 2described above. For example, any number of the projection sections 121and the imaging sections 122 may be provided, and the single section ora plurality of the sections may be provided. That is, the projectionimaging unit 111 is not limited in configuration to the example in FIG.1 and is configured as desired. For example, as illustrated in FIG. 16A,the projection imaging apparatus 101 (projection imaging unit 111) mayinclude one projection section 121 and one imaging sections 122. Also,for example, as illustrated in FIG. 16B, the projection imagingapparatus 101 (projection imaging unit 111) may include one projectionsection 121 and four imaging sections 122 (imaging sections 122-1 to122-4). Also, the projection imaging unit 111 may include componentsother than the projection section 121 and the imaging sections 122.

Also, the projection imaging apparatus 101 may include the plurality ofprojection imaging units 111. In that case, the projection imaging units111 need not be identical in configuration. For example, the projectionimaging units 111 may differ in number of projection sections 121 orimaging sections 122. Also, the projection imaging apparatus 101 mayinclude the plurality of control units 112. In that case, the controlunits 112 need not be identical in configuration. Also, in that case,all the control units 112 may cooperate with each other to handle theprocess described above, or some of the control units 112 may handle theprocess described above.

Also, the projection imaging unit 111 and the control unit 112 may beprovided separately (two different apparatuses). For example, theprojection imaging apparatus 101 may be, for example, configured as asystem that includes a plurality of apparatuses as illustrated in FIG.17.

A projection imaging system 800 in FIG. 17 includes a control apparatus801, a projection imaging apparatus 802-1, and a projection imagingapparatus 802-2. Each of the projection imaging apparatus 802-1 and theprojection imaging apparatus 802-2 is connected to the control apparatus801 in a manner that allows communication through wired or wirelesscommunication. In the case where there is no need to distinguish betweenthe projection imaging apparatuses 802-1 and 802-2, the two will bereferred to as the projection imaging apparatuses 802.

The control apparatus 801 has a similar configuration and functions tothe control unit 112 (FIG. 2) and handles similar processes. Theprojection imaging apparatuses 802 have a similar configuration andfunctions and handle similar processes to the projection imaging unit111. That is, the projection imaging system 800 has a similarconfiguration and functions and handles similar processes to theprojection imaging apparatus 101 that includes one control unit 112 andtwo projection imaging units 111 connected to the control unit 112.

The present technology is applicable to the projection imaging system800 configured in this manner, similarly to the projection imagingapparatus 101. The application of the present technology described inthe first embodiment allows the projection imaging system 800 to controlthe image projection position with more ease.

It should be noted that any number of the projection imaging apparatuses802 may be provided and that there may be only one projection imagingapparatus 802 or two or more projection imaging apparatuses 802. Also,the projection imaging apparatuses 802 may or may not be identical inconfiguration. Also, the projection imaging system 800 may include theplurality of control apparatuses 801. In that case, the respectivecontrol apparatuses 801 may or may not be identical in configuration.Also, the control apparatus 801 may be integrated with any of theprojection imaging apparatuses 802 (that is, may be configured as asingle apparatus). Further, the projection section 121 and the imagingsections 122 may be different units (or apparatuses). Needless to say,the projection imaging system 800 may include apparatuses other than thecontrol apparatus 801 and the projection imaging apparatus 802 describedabove.

Also, the control apparatus 801 and the projection imaging apparatus 802may be connected in the projection imaging system 800 in a manner thatallows communication via a network as any communication network.

Any technique may be used in the network for communication. For example,wired communication, wireless communication, or both thereof may beused. Also, the network may include a single communication network or aplurality of communication networks. The network may includecommunication networks and channels conforming to any communicationstandard, for example, the Internet, a public telephone line network, awireless mobile wide area communication network such as what is called a3G or 4G line, a WAN (Wide Area Network), a LAN (Local Area Network), awireless communication network that allows communication compliant withthe Bluetooth (registered trademark) standard, a communication channelfor short-range wireless communication including NFC (Near FieldCommunication) or the like, a communication channel for infraredcommunication, and a communication channel for wired communicationcompliant with an HDMI (registered trademark) (High-DefinitionMultimedia Interface), a USB (Universal Serial Bus), or other standards,or the like.

4. Note

<Software>

The above series of processes can be performed by hardware or software.In the case where the above series of processes are performed bysoftware, the program included in the software is installed from anetwork or a recording medium.

For example, in the case of the projection imaging apparatus 101(control unit 112) in FIG. 2, the recording medium includes theremovable medium 151 that has the program recorded therein and that isdistributed separately from the projection imaging apparatus 101 todeliver the program to the user. In that case, for example, as theremovable medium 151 is inserted into the drive 145, the program soredin the removable medium 151 can be read out and installed to the storagesection 143.

Also, the program can be provided via a wired or wireless transmissionmedium such as a local area network, the Internet, or digital satellitebroadcasting. For example, in the case of the projection imagingapparatus 101 (control unit 112) in FIG. 2, the program can be receivedwith the communication section 144 and installed to the storage section143.

In addition to the above, the program can be installed in advance to thestorage section, the ROM, or the like. For example, in the case of theprojection imaging apparatus 101 (control unit 112) in FIG. 2, theprogram can be installed to the ROM (not depicted) built into thestorage section 143 or the control section 131, or the like.

<Target for Application of the Present Technology>

Also, the present technology can be implemented as components of allkinds incorporated in any apparatus or an apparatus included in a system(i.e., some components of an apparatus). Examples of such componentsinclude a processor (e.g., video processor) as a system LSI (Large ScaleIntegration) or the like, a module using a plurality of processors orthe like (e.g., video module), a unit using a plurality of modules orthe like (e.g., video unit), and a set with other functions added to aunit (e.g., video set).

Further, the present technology is applicable to a network system thatincludes a plurality of apparatuses. For example, the present technologyis applicable to a cloud service that provides services related toimages (videos) to any terminals including computers, AV (Audio Visual)equipment, mobile information processing terminals, IoT (Internet ofThings) devices, and the like.

It should be noted that systems, apparatuses, processing sections, andthe like to which the present technology is applied is applicable to anysectors including traffic, medical, crime prevention, agriculture,livestock, mining, beauty, manufacturing, home appliances, weather,nature monitoring, and the like. In addition, the application isdetermined as desired.

For example, the present technology is applicable to systems and devicesused to provide content for appreciation purpose and the like. Also, forexample, the present technology is applicable to systems and devicesused for traffic purposes such as traffic condition monitoring andautonomous drive control. Further, for example, the present technologyis applicable to systems and devices used for security purposes. Also,for example, the present technology is applicable to systems and devicesused for automatic control over machinery and the like. Further, forexample, the present technology is applicable to systems and devicesused for agricultural and livestock farming purposes. Also, the presenttechnology is applicable to systems and devices used to monitor naturalconditions such as volcanos, forests, and oceans, and wildlife. Further,for example, the present technology is also applicable to systems anddevices used for sporting purposes.

<Others>

The embodiments of the present technology are not limited to thosedescribed above and may be modified in various ways without departingfrom the gist of the present technology.

For example, the present technology can be implemented as components ofall kinds included in an apparatus or system (i.e., some components ofan apparatus). Examples of such components include a processor (e.g.,video processor) as a system LSI (Large Scale Integration) or the like,a module using a plurality of processors or the like (e.g., videomodule), a unit using a plurality of modules or the like (e.g., videounit), and a set with other functions added to a unit (e.g., video set).

It should be noted that the system in the present specification refersto a set of a plurality of constituent elements (e.g., apparatuses,modules (parts)), and it does not matter whether or not all theconstituent elements are accommodated in the same housing. Therefore, aplurality of apparatuses accommodated in separate housings and connectedvia a network and an apparatus whose modules are accommodated in asingle housing are both systems.

Also, for example, a component described as a single apparatus (or asingle processing section) may be divided into a plurality ofapparatuses (or processing sections). Conversely, components describedabove as a plurality of apparatuses (or processing sections) may becombined into a single apparatus (or a single processing section).Needless to say, components other than those described above may also beadded to each of the apparatuses (each of the processing sections).Further, as long as the components or operation of the system as a wholesubstantially remains the same, some components of a certain apparatus(or a certain processing section) may be included in the components ofanother apparatus (or another processing section).

Also, for example, the present technology can adopt a cloud computingconfiguration in which one function is processed by a plurality ofapparatuses in a shared and cooperative manner via a network.

Also, for example, the above program can be executed in any apparatus.In that case, it is acceptable as long as the apparatus has necessaryfunctions (e.g., functions blocks) to acquire necessary information.

Also, for example, each step described in the above flowchart can beperformed not only by a single apparatus but also by a plurality ofapparatuses in a shared manner. Further, in the case where a single stepincludes a plurality of processes, the plurality of processes includedin the single step can be performed not only by a single apparatus butalso by a plurality of apparatuses in a shared manner. In other words, aplurality of processes included in a single step can be performed asprocesses of a plurality of steps. Conversely, a process described as aplurality of steps can be combined into a single step and performed.

It should be noted that the program executed by the computer may performthe processes of the steps defining the program chronologicallyaccording to the order described in the present specification, inparallel, or individually when necessary as when invoked. That is,unless inconsistency arises, the processes of the respective steps maybe performed in a different order from the order described above.Further, the processes of the steps defining the program may beperformed in parallel to those of another program or combined andperformed together with those of another program.

It should be noted that the plurality of present technologies describedin the present specification can be carried out independently of eachother and alone unless inconsistency arises. Needless to say, any numberof the plurality of present technologies can be carried out incombination. For example, some or all of the present technologiesdescribed in any of the embodiments can be carried out in combinationwith some or all of the present technologies described in anotherembodiment. Also, some or all of any of the present technologiesdescribed above can be carried out together with other technologies notdescribed above.

It should be noted that the advantageous effects described in thepresent specification are merely illustrative and not restrictive, andthere may be other advantageous effects.

It should be noted that the present technology can also have thefollowing configurations.

(1)

An information processing apparatus including:

a correction information generation section adapted to generatecorrection information of an image such that the image is projected ontoa desired region of a real space.

(2)

The information processing apparatus according to (1), in which

the correction information generation section generates the correctioninformation such that the projected image as seen from a given viewpointposition is located at the desired region.

(3)

The information processing apparatus according to (1), in which

the correction information generation section generates a correctionvector for correcting each pixel position of the image as the correctioninformation.

(4)

The information processing apparatus according to (1), furtherincluding:

a region setting section adapted to set the region, in which

the correction information generation section generates the correctioninformation such that the image is projected onto the region set by theregion setting section.

(5)

The information processing apparatus according to (4), in which

the region setting section sets the region in the image.

(6)

The information processing apparatus according to (5), in which

the region setting section identifies the region in the image by using acaptured image of a projection plane including the region.

(7)

The information processing apparatus according to (6), in which

the region setting section identifies a contour of the region in theimage by using a contour point group indicating the contour of theregion in the captured image.

(8)

The information processing apparatus according to (7), in which

the region setting section identifies the contour of the region in theimage from the contour point group of the region in the captured imageby using corresponding points indicating correspondence between theimage and the captured image.

(9)

The information processing apparatus according to (8), in which

the region setting section identifies, for each local portion, thecontour of the region in the image by homographically transforming thelocal contour point group of the region in the captured image on thebasis of the corresponding points corresponding to the local contourpoint group and a corresponding point group in a periphery of thecorresponding points.

(10)

The information processing apparatus according to (9), in which

the region setting section interpolates, by using a curve, the contourof the region in the image identified for each local portion by usingthe contour point group of the region in the captured image.

(11)

The information processing apparatus according to (8), in which

the region setting section detects the corresponding points by using theimage to be projected and the captured image and identifies the contourof the region in the image from the contour point group of the region inthe captured image by using the detected corresponding points.

(12)

The information processing apparatus according to (7), in which

the region setting section detects the contour of the region in thecaptured image and identifies the contour of the region in the image byusing the contour point group indicating the detected contour.

(13)

The information processing apparatus according to (12), in which

the region setting section detects the contour of the region from acaptured image of the projection plane onto which a given image isprojected according to luminance of each pixel.

(14)

The information processing apparatus according to (13), in which

the region setting section detects the contour of the region bybinarizing the captured image.

(15)

The information processing apparatus according to (13), in which

the given image has a uniform luminance level.

(16)

The information processing apparatus according to (13), in which

the given image includes an image all of whose pixels are set to amaximum luminance level.

(17)

The information processing apparatus according to (6), in which

the region setting section corrects distortion of the captured image andidentifies the region in the image by using the corrected capturedimage.

(18)

The information processing apparatus according to (6), furtherincluding:

an imaging section adapted to generate the captured image by capturingan image of the projection plane.

(19)

The information processing apparatus according to (1), furtherincluding:

a projection section adapted to project the image.

(20)

An information processing method including:

generating correction information of an image such that the image isprojected onto a desired region of a real space.

REFERENCE SIGNS LIST

100 Projection imaging system, 101 Projection imaging apparatus, 102Screen, 111 Projection imaging unit, 112 Control unit, 121 Projectionsection, 122 Imaging section, 131 Control section, 201 Projectioncontrol section, 202 Imaging control section, 211 Corresponding pointdetection section, 212 Screen frame position detection section, 213Posture estimation section, 214 Screen shape estimation section, 215Correction vector generation section, 221 Frame detection imagingprocessing section, 222 Lens distortion correction section, 223 Capturedimage frame position detection section, 224 Projected image frameposition identification section, 225 Frame position interpolationprocess section, 231 Viewpoint position estimation section, 232Viewpoint image frame position calculation section, 233 Correctionvector calculation section, 800 Projection imaging system, 801 Controlapparatus, 802 Projection imaging apparatus

The invention claimed is:
 1. An information processing apparatus,comprising: at least one imaging device configured to capture a firstimage of a projection plane which includes a specific region of a realspace; and at least one processor configured to: identify a contour of afirst region in a second image based on a contour point group whichindicates a contour of a second region in the captured first image,wherein the first region and the second region correspond to thespecific region of the real space, and the second image is an imagewhich is to be projected on the specific region of the real space; setthe first region in the second image based on the identified contour ofthe first region in the second image; and generate correctioninformation of the second image such that the second image is projectedonto the set specific region in the real space.
 2. The informationprocessing apparatus according to claim 1, wherein the projected secondimage, as seen from a specific viewpoint position, is located at thespecific region based on the correction information.
 3. The informationprocessing apparatus according to claim 1, wherein the correctioninformation is a correction vector for correction of each pixel positionof the second image.
 4. The information processing apparatus accordingto claim 1, wherein the at least one processor is further configured toidentify the contour of the first region in the second image from thecontour point group of the second region in the captured first imagebased on a plurality of corresponding points indicating a correspondencebetween the second image and the captured first image.
 5. Theinformation processing apparatus according to claim 4, wherein the atleast one processor is further configured to identify, for each localportion, the contour of the first region in the second image based on ahomographic transformation of the contour point group of the secondregion in the captured first image, and the homographic transformationis based on the plurality of corresponding points corresponding to thecontour point group and a corresponding point group in a periphery ofthe plurality of corresponding points.
 6. The information processingapparatus according to claim 5, wherein the at least one processor isfurther configured to interpolate, by using a curve, the contour of thefirst region in the second image identified for each local portion,based on the contour point group of the second region in the capturedfirst image.
 7. The information processing apparatus according to claim4, wherein the at least one processor is further configured to: detectthe plurality of corresponding points based on the second image and thecaptured first image; and identify the contour of the first region inthe second image from the contour point group of the second region inthe captured first image, based on the detected plurality ofcorresponding points.
 8. The information processing apparatus accordingto claim 1, wherein the at least one processor is further configured to:detect the contour of the second region in the captured first image; andidentify the contour of the first region in the second image based onthe contour point group which indicates the detected contour of thesecond region in the captured first image.
 9. The information processingapparatus according to claim 8, wherein the at least one processor isfurther configured to detect the contour of the second region from thecaptured first image, of the projection plane onto which the first imageis projected before the capture of the first image, based on a luminanceof each pixel of the captured first image.
 10. The informationprocessing apparatus according to claim 9, wherein the at least oneprocessor is further configured to detect the contour of the secondregion in the captured first image based on binarization of the capturedfirst image.
 11. The information processing apparatus according to claim9, wherein the first image has a uniform luminance level.
 12. Theinformation processing apparatus according to claim 9, wherein allpixels of the first image are set to a maximum luminance level.
 13. Theinformation processing apparatus according to claim 1, wherein the atleast one processor is further configured to: correct a distortion ofthe captured first image; and identify the first region in the secondimage by using the corrected captured first image.
 14. The informationprocessing apparatus according to claim 1, further comprising aprojector configured to project at least one of the first image or thesecond image.
 15. An information processing method, comprising:capturing, by at least one imaging device, a first image of a projectionplane which includes a specific region of a real space; identifying, byat least one processor, a contour of a first region in a second imagebased on a contour point group which indicates a contour of a secondregion in the captured first image, wherein the first region and thesecond region correspond to the specific region of the real space, andthe second image is an image which is to be projected on the specificregion of the real space; setting, by the at least one processor, thefirst region in the second image based on the identified contour of thefirst region in the second image; and generating, by the at least oneprocessor, correction information of the second image such that thesecond image is projected onto the set specific region in the realspace.